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ABSTRACT 

This report is a comprehensive account of the 
accumulated research on the measurement of quality of effort and its 
siqnificance in understanding the achievement of college students 
with regard to the College Student Experiences questionnaire (Pace, 
1979). Results are based on the responses of 12,000 undergraduate 
students from 40 different colleges over a 3 year period. The 
questionnaire contained 14 quality of effort scales, on wh * ch 
students rated themselves on items- concerned with the use of college 
facilities and the use of personal and social opportunities. The 
other carts of the questionnaire included items that enaDied 
researchers to determine relationships between quality of effort and 
achievement and among many elements that might help to explain those 
relationships. A major conclusion of the study granted the importance 
of all elements that influence "who goes where" to college. The study 
also found that, once the students got to college, what coun i<:* m °^ tt 
was not who they were or where they were but what they did. Quality 
effort appears to be what counts in achieving in college. (JM) 
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Achievement and the Quality of Student Effort 



by C. Robert Pace 
Higher Education Research Institute 
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May, 1982 



This report, prepared for the National Commission on Excellence in 
Education, brings together in. one place a comprehensive account of 
the accumulated research on the measurement of quality of effort and 
Its significance in understanding the achievement of college students. 
The development of the measures and their initial try-out was made 
possible by a grant from the Spencer Foundation, from January, 1970 
through December, 1979. Over the four-year period 1978 through 19B1 
a number of reports about this line of inquiry have been prepared, 
mainly in the form of speeches at professional association meetings 
or for other transient or special occasions. A list of these reports 
is given at the end of the present paper. The present report for the 
Commission provides an overview of previous work and highlights the 
C special relevance of student effort and initiative in the attainment 
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Achievement and the Quality of Student Effort 
By C. Robert Pace 
University of California, Los Angeles 

"If students expect to benefit from what this college or university 
has to offer, they have to take the "initiative, " About 95 percent of 
undergraduates from all over the country agree with that statement." 
Students know that what they get out of college will depend, to a con- 
siderable degree, on what they put into it. They are right. Exactly 
,how right they are, and what it means more specifically for the achieve- 
ment of important educational goals, are the subjects of this paper. 

Excellence, efficiency, productivity, accountability— these are all 
common words in much of the rhetoric about higher education today. But 
more often than not, that rhetoric has been one-sided. It assumes that 
leaving cgllege before getting a degree is a sign of failure, when in 
many cases it may be a prudent and well-informed decision. It assumes 
that professors produce learning. It^assumes that the college, not the 
national economy, controls the job market, it assumes that if you don't 
benefit from college, it's their fault. It assumes that the student is 
buying a product and is therefore entitled 4 to a value for that product. 
It is a curious line of thinking because actually the student at a later 
point in time i_s the product! 

Colleges are, of course, accountable for a lot of things. They are 
accountable for the resources and facilities, the programs and procedures 
the stimuli and standards they provide for student learning and develop- 
ment. But surely the students are also accountable for the amount, 



scope, and quality of effort they invest in their own learning and 
development, and specifically, in using the facilities and opportunities 
that are available in the college setting. Accountability for achievement 
and related student outcomes must consider both what the institution 
offers and what the students do with those offerings. 

What is quality of ef ? ort? How is it measured? Having measured 
it, what do we know about its importance in accounting for quality of 
results? Then, although our current data come from higher education, 
what relevance might be projected for the quality of education at all 
levels? 

All learning and development requires an investment of time and 
effort by the student. Time is a frequency dimension. Effort is a 
quality dimension in the. sense that some kinds of effort are potentially 
more educative than others. Effort at what? The college experience 
consists of the events that occur in a college environment.. The relevant 
experiences are ones that stem from events and conditions and facilities 
which the college makes possible, and which are intended to facilitate 
student learning and development. The most salient of these events and 
experiences are clustered around a number of fairly common behavior 
settings. A behavior setting is a place, a physical setting, in which 
certain types of activity typically occur- Obvious examples of such 
facilities in a college or university are classrooms, libraries, labora- 
tories, cultural facilities, student unions, athletic and recreational 
facilities and residence units. Then there arfe other events and experi- 
ences which are not necessarily connected with a specific facility, but 
are nevertheless important opportunities for personal and social develop- 
ment. Obvious examples are contacts with faculty members, involvement 



in clubs and organizations, experiences in writing, the breadth and 
depth of student acquaintances, opportunities related to self-understandina 
and the general nature and level of student conversations. The facilities 
and opportunities just listed are the ones we selected for measuring 
quality of effort in their use. 

How does one measure quality in the use of such facilities and 
opportunities? We devised brief activity checklists for each topic, to 

which students respond by indicating how often they have engaged in each 

... / 
activity during the current school year. The activities rSpge from 

relatively common ones that require little effort to ones that require a 
greater investment of effort and that have a greater potential for 
influencing learning and development. Some examples will clarify the 
concept and the measurement of it. 

" Consider the library as a facility. A library has certain purposes 
and certain properties as a repository and a resource. To what extent , 
do students use it to capitalize on those properties? Some students may 
use the library building simply as a convenient and quiet Dlace to study 
materials they have brought with them. This usage of the facility hafe 
no basic connection with its purposes as a library. Others may use the 
library to read sbmething that was explicitly assigned*. This makes some 
use of the library as a repository. Still other students may discover 
that the library is a resource offering exciting avenues for exploration- 
examining indexes and guides, following up on various references, looking 
for materials under different headings, browsing in the stacks and' 
taking out something because it looked interesting and having that lead 
on to something else. The quality of effort dimension with regard to 
library experiences is that of using the potential which a library has 
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' to offer — greater degrees of independent exploration, learning how to 
find information one needs, and thereby increasing one's competence for 
independent learning* 

With respect to course learning, such activities as flaking outlines 
from classnote^ and readings or trying to explain the material to another 
person are higher level cognitive activities than merely taking notes or 
underlining points in a textbook. The quality dimension is the level of 
cognitive effort, with the higher levels contributing more solidly to 
the acquisition of knowledge and understanding. v 

For science laboratory, the underlying quality dimension runs from 
rather routine tasks and preparation to activities' that involve efforts 
to improve laboratory skills, and more independent use of lab facilities* 
For the student union the check list of activites range from casual and 

informal use to more programmatic use such as attending events and 

t 

participating 1n meetings. ' For residence units the^quality dimension 
runs from general socializing to more personal exchanges involving such 
activities as helping and sharing and studying together and working on 
projects. In the use of cultural facilities (art, music, theater) the 
activities range from attending arid talking about to efforts toward 
greater understanding as might come from seeking the views of experts 
and critics and from personal involvement. The checklist of activities 
for athletic and recreational facilities goes along a dimension from 
generally informal use as in exercise and games to greater efforts 
toward improvement and skilled performance. 

The checklist of efforts to make contact with faculty members out 
of class ranges from fairly routine and casual— asking for information 
about assignments, or just saying hello or visiting informally after 
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class - to more serious contacts such as talking about ideas for a term 
paper or project, discussing career plans and ambitions, inviting criticf^ 
and even discussing personal problems or concerns. To do these latter 
things requires more initiative, more effort, and may be mare educative. 
With respect to clubs and organizations the activities range from awareness 
of events and organizations to attending events, discussing programs, 
working on a committee, etc. For experiences in writing the underlying 
quality dimension runs from a general concern with words and grammar and 
revision, pretty much by onreself , to actively seeking criticism from 
others and a greater concern for clarity and style. Activities related 
to self-understanding range from general curiosity about one's own 
behavior and others to more focused and expertly informed sources of 
self-understanding which one might get from reading articles or books, 
or taking a test to measure one's abilities or interests, or talking 
with a counselor or some other specialist'about problems. With respect 
to student acquaintances the scale runs from breadth of acquaintances to 
greater depth of acquaintances involving serious conversations with 
people who differ a lot from oneself. The scale called Topics of Conversa- 
tion ranges frOiTi personal and interpersonal topics of fairly immediate 
experience to intellectual and cultural topics concerning values and 
social issues. And finally, the scale called Information in Conversa- 
tions measures the extent to which knowledge and expertise and persuasion 
are brought to bear in. those conversations. 

Overall, the checklists or scales are intended to provide a systematic, 
structured, and reliable inventory of the amount, scope, and quality of 
effort students put into capitalizing on the college experience. To 
each activity in these various categories, the student responds by 
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indicating "never, 11 "occasional ly/^&ften," or "very often." Never 
gets one point, occasionally gets two points, often gets three, and very 
often gets four. The student's score on a particular scale is simply 

' the sum of those responses. A high score cap only be obtained- by' engag- 
ing in the higher level activities, artel therefore the score reflects the 
quality of effort and experience and not merely its frequency. With 
two minor exceptions, each scale consists of ten items. Subsequent 
analyses have demonstrated that these scales are good measures. They 

.are highly reliable. ATI the items are discriminating. And each scale 
consists primarily of a single hierarchial dimension that ranges from 
low to high quality of effort with respect to the topic. These quality 
of effort scales may be thought of as analogous to a battery of achieve- 
ment tests. The former measure what students put into their education; 
the latter measure what they get out of it. 

The fourteen quality of effort scales - seven concerned with the 
use of college facilities and seven concerned with personal and social 
opportunities—form the major part of a questionnaire called C ollege 
Student txperiences (Pace, 1979). The other parts of the questionnaire 
include items that enable one to determine relationships between quality 
of effort and achievement, and many elements that might help to explain 
those relationships. 

The first part of the questionnaire consists of a series of items 
under the heading of "Background Information." There are two types of 
information in this section - first, information about the status of the 
individual, namely age, sex, marital status, race, and educational level 
of the parents; and second, information about the status of the individual 
in college such as year in college, whether one lives in the dormitory 



or elsewhere, grades, major field, expectations about continuing for an 
advanced degree, amount of time spent on academic activities, amount of 
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time, if any, spent working on a job, and the proportion of cbllege 
expenses paid by parents or family. There is also an index of-students* 
satisfaction with college consisting of the following .two questions: 
"How well, do you like college?" and "If you cQiuld start over again wpuld 
you go to the same college you are now attending?" 

Another part of the questionnaire is called "The College Environment 
This consists of eight rating scales each dealing with a different 
aspect or characteristic of the college environment. The first four 
ratings ask students to indicate how^uch emphasis they feel is given at 
the college to certain goals or aspects of student development—emphasis 
on the development of academic, scholarly^ and intellectual qualities; 
emphasis on, the development of esthetic, expressive, and creative qualitic 
emphasis on being critical, evaluative, and analytical; and emphasis on 
the development of vocational and occupational competence. For each of 
these emphases there is a seven*point rating scale, ranging from strong 
emphasis at one end to weak emphasis at the other. Then there are 
three rating scales that refer to personal relationships within the 
college environment—relationships with other students, relationships 
with faculty members, and relationships with administrative -offices and 
officials. The student is asked to characterize those relationships at 
h,is or her college, again on seven point scale. The relationships with 
students range at one end from friendly and supportive, to uninvolved 
and alienated at the other end. The relationships with faculty range 
from approachable, 4 helpful, and understanding at one end of the scale to 
remote, difficult, and impersonal at the other end. With administrative 



officials the scale ranges from helpful and open minded to discouraging 
and unsympathetic. Finally, there is a scale described as general style 
of operation as an organization, ranging from flexible, adaptive, and 
.considerate to rigid, resistant, and bound by regulations. 

The rating scales are intended to capture in a brief fashion basically 
important qua'litTes .of the colleges environment: first, the extent to* 

which it emphasizes certain objectives o'r goals, second, the nature-and 

ft 

quality of personal relationships within the environment and particularly 
the supportiveness of those relationships, and finally, the style of 
operation of the organization as studepts perceive it. 

The final section of the questionnaire is called "Estimate of 
Gains. 11 This consists of eighteen statements of fairly typical and 
imL~. .ant objectives, such as vocational training, a broad general 
education, writing clearly and effectively, ability to think analytically 
and logically, and so forth. For each of these eighteen goals the 
student is asked: To what extent do you feel y.ou have gained or made 
progress (in college up to now)?. The student can answer by indicating 
"very little," "some," "quite a bit," or "very much." These self-reported 
gains can be regarded as an indication of the extent to which students 
believe they are achieving important objectives of higher education; and 
one can then determine the extent to which high quality effort contributes 
to high attainment or progress toward related goals. 

Since all of our data come from student responses to a question- 
naire, we are dependent on the credibility of student self-reports. We 
have not objectively measured students attainment; wq have askect them to 
indicate how much gain or progress they feel they have made toward the 
attainment of various goals. We have not directly observed, or filmed, 



or recorded how often\students engaged in, various activities; we have 

asked them to indicate how of,teri they have done so. Can we believe what 

they say? A great, deal -of social science research, and allvquestibnr 

• nai re surveys, are based on self-reports. Fortunately, there is ample 
•> " - ' ' * " 

evidence to support the credibility of such data, especially of /the type 

« " r "» * ' . ' 

of data from the College Student Experiences questionnaire. "Here'are a 

'few examples from past research: comparisons of students self-reported 

grades with. Registrar's records show correlations qf :9CTand higher; 

self-reports by adults of many kinds of factual data (home ownership, 

have library -card, etc.,) are typically 96% to 95% t accurate; student 

reports of past achievements (in athletics, leadership, music, speech, 

drama, art, writing, science) are highly dependable; adults self-reports 

of' activities engaged in "during the past year 11 (in political and civic 

affairs, cultural affairs, religion, and science) were found to be 

85% identical with their reports six months later. In short, there is 

no reason to doubt the accuracy of student answers to the background 

information items in the College Student Experiences questionnaire; nor 

is there any reason to doubt their response to the activities- in the 

quality pf effort scales when they say they have "never" engaged in the 

activity. Some, in trying to recall their activities during the school 

year, may forget about certin ones— the responses 'may be 85% to 95% 

accurate instead. of 100%; but when activities are reasonably- specif ic 

and clearly described and refer to things students easily recognize, 

then their responses, based on past research, can be accepted as quite 

accurate an^J therefore credible. < ~ 

There is another aspect of credibility which refers to the meaning 

of certain responses. We can believe it when they say they have or have 



not. engaged in a particular activity,; but if .they have, what do they 

"mean by^'occasionally," "often/ 1 and "very often? 11 While it is obvious 

, that these are clearly' different frequencies, how do students decide 

which of the threje choices is appropriate for therrf? Judgments. of this 

kind are typically ma.de with some reference group in' mind; and presumably 

the reference grottp would most l.ike-ly be other students they know. So, 

"i * • * 

does "often' 1 at Amherst mean the same as "often" atr UCLA? * Answers to 

.this and related matters have recently been reported (Pace and Friedlander, 

1981) as follows: although there is considerable overlap between what 

^s meant by occasionally, tiften, and very often, there is a]so a clear 

concentration of responses yithin each category^ and a clear direction of 

greater frequencies, (i.e., lumber of times the activity has been engaged 

in) as\one moves from occasional to often to very often; but the major 

differences in the meaning, of the response categories are ones related 

to the topic or specific activity and that,, given the same topic, differ- 

ences between colleges and between different groups of students within 

colleges are minor. 

A similar i-ssue of interpretation exists in relation to students 1 ^ 

s£lf-estimates of gains on the eighteen statements of goals. Evidence 

to support the conclusion that these estimates of progress are accurate 

and va/id is very convincing in those cases where objective comparative 

data are available. . For example, we can accept as 100%, or very nearly 100%, 

accurate students identification of their major, field. They surely know 

whether they are majoring in ehgineering or . English or economics! We 

also know, from decades of records and requirements .for graduation* that 

the major field, whatever it is, is the field in which they take more 

courses, and more courses at an advanced level, and spend more time at^_ , 
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than any other field. . Since we know this is true, the results on the 
quality of effort scales should reflect that truth. They do. The mean 
score for majors in humanities and in fine arts on involvement in art- 
jnusic-theater activities is much much higher than the m$an score of any 
other group. So also, the mean score on the Science Lab scale for 
majors in natural sciences is. much much higher than the mean score of 

any other group. We also know, from several decades of data 6n comprehen- 

\ 

sive achievement tests such as the Area Tests of th$ Graduate Record 
Examinations, or the Undergraduate Assessment Program ^of Educational 
Testing Service, and The Advanced GRE Tests and the Major Field Tests of 
ETS, that students make their highest scores on those tests, or parts of 
tests, that are most closely related to their major field, or to their 
"area of interest. 11 Since we know^this is true, the results on the 
self-estimates of gains should reflect that truth. They do. Toward the 
objective "developing an understanding and enjoyment of art, music, and 
drama," 85% of the majors in fine arts felt they had made very much or 
quite a bit of progress. This percent is three times greater than the 
percent for science majors or social science majors. Toward the objective 
"broadening your acquaintance and enjoyment of literature," three-fourths 
of the humanities majors reported very much or quite a bit of progress 
compared with one-fourth of the science majors. Toward the objective 
"understanding the nature of science and experimentation," 84& of the 
science majors reported very much or quite a bit of progress, compared 
with 11% and 15% respectively for fine arts and humanities majors. 

In our total set of data there^ e^e many other examples of congruent 
or validating relationships between known facts and students' ratings. 
Consequently, we can accept the self-reports of activities and the 
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self-estimates of progress as broadly credible, valid, and true to the 

facts. We have digressed to make these points because one often encounters 

peptics who dismiss the results of self-reports as unreliable and hence 

unacceptable. That skepticism, in the case of the present data and of 

* 

many other questionnaire^ surveys, is without foundation. Having made 
the point that there are solid grounds for claiming the accuracy and 
reliability of the evidence, we will now present that evidence, 

"fey the end of spring 1981, about 12,000 undergraduates from 40 different 
colleges and universities had responded to the College Student Experiences 
questionnaire. This represents 4,000 for 11 colleges and universities 
in 1979, 5,000 from 19 schools in 1980, and 3,000 from 13 schools in 
1981. Three of the schools were second-time users, so that the number 
of different schools is 40. In the spring of 1982, 30 schools are using 
the questionnaire and an additional 6,000 or more student responses are 
anticipated. With the exception of a few places where the questionnaire 
was given to a special group--for example, freshmen only, seniors only, 
etc. --the sample of student responses comes from a good cross-section of 
undergraduates. The 40 colleges and universities, classified in accord 
with the Carnegie^ Council 's classification system, include 8 doctoral 
granting universfties, 14 comprehensive universities and colleges 
(6 public and 8 private), 9 Type I liberal arts colleges, and 9 Type II 
liberal arts colleges. Comprehensive colleges and universities have a 
greater range of offerings than liberal arts colleges and also offer 
masters or first professional degrees in one or more fields, but do not 
offer doctoral degrees or advanced professional degrees. The difference 
between Types I and II liberal arts college, is in the academic selectivity 
of the students, with Type I being the more highly selective. 
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The first set of results, based on an analysis of some 3,000 student 5 
responses at 11 colleges and universities' in 1979, is, in nany respects, 
the most dramatic. Subsequent analyses based on larger samples have 
. merely confirmed these results. The question to be answered is this: 
given all the elements in the questionnaire—students background charac- 
teristics, their status in college, their satisfaction with college, 
their assessment of the college environment, and their scores on, the 
various quality of effort (QE) scales— what best predicts their achieve- 
ment with respect to the list of goals of higher education? For this 
analysis, the set of 18 goals has been grouped into four broad catagories, 
as follows: 

1. Personal and Interpersonal Understanding 

Developing your own values and ethical standards 

Understanding yourself— your abilities, interests, and 
personality 

Understanding other people and the ability to get along 
with different kinds of people 

Ability to function as a team member 

Developing good health habits and physical fitness 

2. General Education Objectives 

Gaining a broad general education about different fields 
of knowledge 

Developing an understanding and enjoyment of art. music, 
and drama 

Broadening your acquaintence and enjoyment of literature 
Writing clearly and effectively 

Becoming aware of different philosophies, cultures, and 
ways of life 

ERIC . 
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3. Intellectual Competencies 

Ability to think analytically and logically 

Ability to put ideas together, to see relationships, 
similarities, and differences between ideas 

Ability to learn on your own, pursue ideas, and find ♦ 
information you need 

Acquiring background and specializations for further 
education in some professional, scientific, or. 
scholarly field 

4. Understanding Science 

Understanding the nature of science and experimentation 

Understanding sci'entif* nd technical developments and 
their applicatic m 1 society 

Quantitative thinking - understanding probabilities, 
proportions, etc. 

Students reported progress toward each of the objectives in these four 
groupings were addejl up to give an estimate of progress toward the more 
general objective defined by the group category. These four categories 
of achievement are then used as criteria of attainment, to be predicted 
or explained by all the information we have about students, environments 
and quality of effort. The statistical procedure is called stepwise 
multiple regression. Thvs simply means that the computer program first 
identifies the variable that has the largest relationship with the 
criterion, then the variable that has the next largest relationship, 
etc., until adding more variables contributes little or nothing more (1% 
or less) toward accounting for the performance on the criterion. 

Here are the results from this analysis showing the relative magni- 
tude of relationships or predictions of the criteria. 
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Achievement 



Multiple 



Personal/Interpersonal Understanding 

QE: self understanding .38 

QE: athletic facilities ^48 

Environment: student relationships .54 

Satisfaction with college .56 

QE: conversation topics .57 

All other variables .62 

Intellectual Competencies 

QE: course learning .36 

Satisfaction with college .45 

Year in college ~ .49 

Environment: critical emphasis .54 

QE: conversation level .56 

QE: science laboratory .58 

Environment: student-faculty relationships .59 

All other variables .63 
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Multiple. R R^ 

General Education Objectives 

QEr art, music, theater ,43 ,18 

QE: writing .50 ■ . -25 

Environment: critical emphasis ,55 " ,30 

Environment: student-faculty relations ,57 .33 

QE: conversation topics .59 .35 

Major field: Fine Arts - Humanities .61 .37 

All other variables .65 .42 



Understanding Science 



QE: Science laboratory 


. -52 


.27 


Major field: natural sciences - math 


. 56 


.31 


Sex (male) 


.59 


.34 


QE: course learning 


.61 


.37 


Major field: humanities - fine arts (negative) 


.63 




All other variables 


.68 


.46 



What these figures show is quite clear. In relation to every one 

of the four main categories of achievement, one or more of the quality 

of effort scales (QE) makes the greatest contribution toward explaing 

2 

that achievement. The numbers under the column labelled R are really 
percentages—that is, they show the percent of the variance on the 
criterion that is "accounted for, 11 or more simply, M when you know these 
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things this is how much of the result you have been able to explain or 
predict." 

Another way to highlight the contribution that quality of effort 
uakes in predicting achievement is to put all the variables into the 
computer in a predetermined sequence: first, put in all the students 1 
background or status variables; second, put in all the college status 
variables; third, put in all the environment ratings; and finally, after 
all these commonly utilized variables have contributed as much as they 
can to explaining achievement, put in the quality of effort variables to 
see whether they add anything to explaining the achievement. 
Here are the results of that oattern of analysis: 



Achievement 


Multiple R 


£? 


Personal/Interpersonal Understanding 






Student status variables 


.36 


.13 


College status variables 


. .47 


.22 


Environment ratings 


.49 


.24 


Quality of Effort Scales 


.62 


.39 


Intellectual Competencies 


Multiple R 


R? 


Students status variables 


.10 


.01 


College status variables 


.53 


.28 


Environment ratings 


.55 


.30 


Quality of Effort Scales 


.63 


.40 
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Multiple R R 2 



General Education Objectives 

Student status variables .14 .02 

College states variables .48 • .23 

Environment ratings .55 .30 

Quality of Effort scales- " .66 .43 

Understanding Science 

Student status variables .23 .05 

College status variables .56 .34 

Environment ratings .60 .36 

Quality of Effort scales .68 -47 

These data show that, before considering the quality of effort 
measures, one can account for somewhere between 24% and 36% of the 
result on the criterion. This is almost "exactly what many past studies 
have shown. But, when the qualify of effort measures are added, one can , 
now explain from 39% to 47% tff w,i* performance on the criterion—a 
substantial increase in our understanding, from 10 to 15 percentage 
points better than past research has typically been able to explain. 

The results just shown lead to a very significant conclusion, one 
that differs from much prior research which has held that student character- 
istics and family background are the most important determinants of 
achievement. The new conclusion is thio: granted the importance or all 
the elements that influence who goes where to college, once the students 
get there what counts most is not who they are or where they are but what 
they do . Prior research has not included what turns out to be the 
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most influential variable—the quality of effort that students themselves 

invest in using the facilities and opportunities for learning and develop- 

went that exist in the college setting. Now that "quality of effort" 

• has been included, better explanations and new conclusions emerge. 

... * 

In addition to the above general relationship between effort and 

attainment, there are some analyses ;that show the special diagnostic 

significance of quality of effort and lead to further refinements in 

prior conclusions. 

It's true, for example, that gains in academic/intellectual competen- 
cies are related to students 1 grades— the better the^grades, the larger 
the gains. But that's not the whole truth. The wh^le truth is that 
students who have high scores on the quality of effort scales related to 
academic/intellectual experiences (course learning, library, writing, 
and contacts. With faculty) make much greater gains than students whose 
quality of effort scores are low, regardless of their grades. In fact, 
B-^udents with high quality of effort scores make more progress than 
B+ students with low quality of effort scores. 

It's true that living on campus versus living at home or elsewhere 
is positively related to students' satisfaction with college. But it is 
not the whole truth. The whole truth is that freshmen who live in the 
dormitory or fraternity /sorority but put a low quality of effort into 
using the residence facility might as well have stayed at home so far as 
satisfaction |with college is concerned. 

It's^true that students who expect to continue their education in 
graduate or^prpfessional school have higher gains scores on academic/ 
intellectual competencies and higher effort scores on academic/intellectual 

s 

experiences than students who do not plan to continue beyond the bacheTor's 
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degree, and that outcome scores and 'eff ort scores are typically greater 
as one moves from freshman to senior year. But again, that's not the 
whole truth. The whole truth i§ that, for every year. in college, students 
who do not plan to continue but nevertheless have high quality- of effort 
scores, make higher scores on the outcome measures than students who do 
plan to continue but have low- quality of effort scores. 

It's true that "time on task 11 has been shown in many research 
studies to be a very important factor in explaining achievement. But 

o 

compared to quality of effort, time on task is a relatively weak explana- 
tion. In the present study, two definitions are similar to the idea of 
"time on task. 11 One Is how long the students have been in college; th? 
other is how many hours e week the students usually spend on activities 
related their school work. Our analyses tonfirm the importance of time, 
but also the greater 'tfportanee of effort. It's true, for example, that 
gains on outcome measures related to intellectual competencies and 
to generi education are related to how long one has been in college—the 
gains reported by seniors are significantly greater than the gains 
reported by freshmen. But the whole truth is that freshmen whose quality 
of effort scores for intellectual/academic experiences are high (above 
average) report greater gains in intellectual competencies and in general 
education than juniors or seniors whose quality of effort scores are low 
(below average). It is also true that sheer time spent on academic work 
(number of hours a week) is related to progress toward objectives related 
to general education, to intellectual comptencfes, and to grades. But 
the whole truth is that students who spend a lot of time at a low level 
of qua 1 ity. make less progress than students who spend fewer hours at a 
high level of quality; and students who spend about 40 hours a week of 
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high-quality effort get better grades than students who spend 50 or more 
hours of low quality effort. 

These relationships between time, quality of effort, grades, alnd 
gains are documented in the next two sets of figures, based on-more than 
7700 students f rom „30 colleges and universities. 
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Length of Time in College Vs. Grades and Gains 



Year in College "aW "Quality — . 

of Academic/intellectual . ' Gains in Gains in 

Experiences Grades Intellectual Competences General Education 

Freshmen 2.9 10.6 11.7 

Low QE 2.8 * 9.8 10.8 

High" QE 3.1 11.5 12.9 

Sophomores 3.1 11.5 12.3 

Low QE 3.0 10.7 11.1" 

High QE 3.2 12.3 • 13.5 

Juniors 3.2 11.8 12.3 

Low QE 3.0 10.9 . 11.3 



< High QE . 3.4 12.5 13.2 

Seniors 3.3 12^1. " * 12.5 

Low QE - 3.1 • 11.3 11.4 

High QE 3.4 12.7 13.3 



N = 7720 



Note: Grade point averages are 2.0 = C; 3.0 = B; 4.0 = A. Mean scores on the Intellectual 

Competence and General Education objectives that are different from one another by .3 dr 
more are statistically significant. « 
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Hours, Per Week on Academic Activities Vs. Grades and Gains 



Hours Per Week and Quality of 

Academic/Intellectual- Gains in * - Gains in 

Experiences . Grades Intellectual Competence General Education 

About 20 hours or less 2.9 10.6 11.2 

Low QE 2.8 10.0 • 10.5 

' High QE 3.0 11.8 12.6 

About >30 Hours 3.0 11.3 12.2 

Low QE 2.9 10.6 . 11.3 

High QE 3.1 12.1 13.2 

n 

About 40 hours * 3.3 11.9 12.7 

Low QE , 3.2 11.1 " 11.6 

High QE „ 3.4 12.4 13.4 

About 50 hours or more 3.5 12.3 12.6 

Low QE 3.3 11.5 11.2 

High QE 3.6 12.8 , 13.3 
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* Not only does quality of effort have a general predictive value/ 
/ and a special diagnostic value, as the results thus farj?resented have 
shown, it also hai$*what one might call a pervasive value. By this we 
• mean that the range or scope of high quality effort is>elated-to the 
range or scope of high achievement. The more aspects of the college 
experience, (use'of faci 1 ities* and opportunities) one participates in at 
an above average level of quality of effort, the more objectives (different 
goals of higher education) one makes above average progress in their 
attainment. Breadth of involvement and breadth of attainments go hand 
in hand. 

Of the 14 quality of effort scales in the questionnaire, 12 are 
answered by everyone (not all students live in a campus residence facility 
and so do not respond to the Residence scale, and not all students have 
had a science laboratory course and so do not respond to the Science ^ 
Laboratory scale. Of the 12 scales' applicable to everyone, four are 
mainly concerned with academic/intellectual activities (course learning, 
library, faculty, and writing),, four are primarily personal and inter- 
personal (personal experiences, student acquaintences, conversation 
topics, and conversation level), and four are primarily centered around 
group facilities and, associations (student union, clubs and organizations, 
athletic and recreational facilities., and cultural facilities related to 
art, music, and theater). We devised a "breadth index" which is defined 
as* the number of scales (different aspects of campus life) on which a 
student's score is above the'median of some baseline group. .This baseline 
could bet\the median at one's cwri institution, or the median of all 
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student responses at all institutions. Of the 30 colleges and universities 
from which data had been obtained by the spring of 1980, 24 were selected 
to form a multi-institutional baseline, chosen* because each of them had 
obtained replies from a good cross-section of, students* Students 1 
scores on the breadth index could range from 0 to 12, and in fact do so. 
Some students invest above average quality of effort on all twelve of 
the topics and some students invest above average quality on none of the 
twelve topics. The distribution of breadth scores for 7800 student* at 
these 24 colleges and i» iversities was a noriral distribution^ one fourth 
(25%) of the students had a breadth score of 9 or higher; and about the ' 
same proportion (27%) had avbreadth score of 3 or lower; and the median 
breadth score (6.4) was almost exactly halfway between 0 and 12. 

Using a breadth score of 9 or higher (the upper fourth) as a defi- 
nition of "high breadth," large differences between one college and 
another were revealed. For example, at one college only 10% of the 
student body had a brekdth of 9 or higher; whereas at another college 
61% of the students had a breadth score of 9 or higher. Clearly at some 
colleges the vigor and vitality of wha,t students put into the college 
experience covefi^mucK wider range o* activities, is much more pervasive, 
than is true at other colleges, Indeed, the breadth index for a college 
may be a good indicator of \'he quality of its undergraduate education 
program, or at least of the quality of undergraduate student experience 
at the college. 

Evidence that the breadth score might be a good index of the quality 
of undergraduate education on the campus is suggested by the relationship 
between breadth scores and outcomes. The rank order correlation, of the 
24 institutions studied, between breadth of effort and breadth of outcomes 
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is ,80, Breadth of outcomes is the number of outcomes or objectives in 
which the institution's score (percent of its students reporting very 
much or quite a bit of progress) was at* or higher than the score for the 
composite of all institutions in the study. So, the breadth of above 
average effort was clearly associated with the breadth of above average 
attainments — the broader the scope 6f effort the broader the range of 
outcomes. 

Other evidence of the relation between breadth* scores and attain- 
ment is shown in the following figures—comparing the percent of students 
with low breadth scores vs. high breadth scores with respect to their 
reported progress toward each of 17 objectives. 
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RELATIONSHIPS BETWEEN BREADTH SCORES 
AND ATTAINMENT 



Percent of students reporting 
"quite a bit 11 or "very much" progress 
toward various objectives 



Objectives 



Among students with 
low breadth scores 
(0-3) 



Among students with 
high breadth scores" 
(9-12) 



Goals related to general 
education 



breadth of knowledge 
art, music, theater 
literature 
writing 

philosophies and cultures 

Goals related to intellectual 
competence 

specialization 
analytical thinking 
ability to synthesize 
ability to find information 

Goals related to science 

understanding science and 

experimentation 
understanding science and 
technology 
O mantitive skills 

ERJC 



52 
14 
14 
33 
37 



51 
50 
53 
61 



27 

28 
35 



80 
48 
55 
69 
78' 



74 
78 
87 
91 



41 

42 

53 
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Percent of students reporting 
"quite a bit" or "very much" progress 
toward various objectives 



Objectives 



Among students with 
low breadth scores 
(0-3) 



Among students with 
high breadth scores 
(9-12) 



Goals related to personal 

and interpersonal understanding 

clarifying values 
self -understanding 
understanding others 
function as a team member 
develop good health habits 
and physical fitness 



44 

58 
54 
30 

22 



85 
89 
89 
66 

51 
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On all objectives the percent of students reporting very much or 
quite a bit of progress is much greater among the students with high 
breadth scores. On many objectives the percentage for the h.igh breadth 
group is more than twice as large as the percentage for the low breadth 
group. This again suggests the pervasive value of the concept of quality 
of effort, for it has an influence on every one of the objectives. 

In a local study, based on the responses of a good cross-section of 
undergraduates at UCLA, interrelations among environment, effort, satis- 
faction, and attainment were explored. These interrelationships further 
illustrate both the predictive and the pervasive value of quality of 
effort. For the UCLA study we used students satisfaction with college 
as the criterion. It seems reasonable to expect that people who don't 
.like what they'are doing and don't like where they are probably are not 
putting much effort into the activity or getting much out of 1t*. The 
two questions asking about students satisfaction had the following 
answers at UCLA: 

How well do you like college? 



If you could start over again, would you go to the same college you 
are now attending? 



By giving 4 points to the most favorable response, 3 points to the next 
most favorable response, etc. for each question, and adding the points, 
the following distribution was obtained: 



I am enthusiastic about it 

I like it 

I am more or less neutral about it 

I don't like it 



38% 
'46% 
12% 
3% 



Yes, definitely 
Yes, probably 
Probably, no 
No, definitely 



38% 
43% 
14% 
4% 
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Satisfaction score 



8 Dnintc 


22% 


verv sati sf i ed 
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7 nnints 

# 1 II 


27% 




6 points 


28% 


satisfied 


5 points 


13% 


neutral to negative 


4 points 


7% 




3 points 


■2% 




2 points 


1% 





The meanings of scores 8 or 7 at the most favorable end of the 
scale, and of scores of 2 or 3 at tf\e most unfavorable end ara clear. 
Students who are very satisfied with their experience at UCLA comprise 
49% of the group, in contrast to 3% who are very dissatisfied. A score 
of 6, obtained by 28% of the group, can be charactarized as "satisfied," 
for it nearly always consists of people who said "I like it" to the 
first question, and "probably yes" to the second question. Anyone 
getting a score of 5 or lower must have answered at least one question 
negatively or neutrally. Parenthetically, college students across the 
country appear to be rather well satisf ied—the results from 10,000 
undergraduates at 40 colleges and universities showing 42% in the "very 
satisfied" bracket and 4% in the "very dissatisfied" bracket. 

In the UCLA study we divided the students into thre£ groups, as 
defined above - very satisfied, satisfied, and neutral to negative— and 
then compared the, responses of these three groups with respect to the 
quality of effort scores on all 14 aspects of the college experience, 
their characterizations of the college environment on all of the environ- 
ment ratings, and their ratings of progress toward all of the objectives. 
On every quality of effort scale, and on every characterization of the 
environment, and on reported progress toward every objective, the highest 
(most favorable) mean scores were made by the students who were "very 
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satisfied 11 with UCLA, the next best scores were made by students who 
were "satisfied," and the lowest scores were made by those who were 
"neutral to negative." There were no exceptions to .this pattern. 

Students who are most satisfied with college put the most-into it 
and get the most out of it. Using satisfaction with college as the 
criterion, and then determining which variables of all the ones included 
in the questionnaire have the highest relationship to that criterion 
(best predict or best explain it), the two most influential variables 
,were as follows: first, students gains in the group of objectives we 
have described as intellectual competence, and second, environment in , 
which relationships among students* were characterized as friendly and 
supportive. So, when students are very satisfied they believe they are 
developing their intellectual powers, and find the environment to be 
friendly and supportive. Since we don't really know the direction of 
these relationships we can also state the generalization another way: 
when students are making progress in the development of their intel- 
lectual powers, and when their experience in the environment is charac- 
terized by friendly and supportive relationships with other students, 
they are very satisfied with college. 

These results, together with ones presented earlier, reveal a 
circle of influence on what one can surely call excellence: high quality 
effort is the best predictor of high quality achievement; high quality 
achievement in intellectual powers is the best predictor of high satisfac- 
tion with college. And satisfaction as well as achievement is further 
enhanced in an atmosphere that is friendly and supportive. 

Another local study at UCLA (Lara, 1981) of community college 
transfer students included in it some comparisons between the quality of 
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effort at UCLA of transfer students who had persisted and those who had 
dropped out, and also the quality of effort of these two groups when 
they were in community collegje. The academic quality of effort scales 
were used in this study— course learning, library, writing,- and faculty 
contacts. The population included all community college transfers to 
UCLA in the fall of 1977. Sixty one percent of them were located and 
responded to a questionnaire in the spring of 1979—824 respondents were 
still- enrolled at UCLA and 312 had dropped out. The students indicated 
how often they had engaged in the various activities at UCLA and also 
how often they had engaged in those same activities when they were in 
community college. Other parts of the questionnaire asked about some 
environment characteristics at community colleges and at UCLA, and 
progress toward certain objectives at community college and at UCLA. ,0n 
the quality of effort scales for librabry, writing, and course learning, 
the scores of both the dropouts and the persisters were higher at UCLA 
than they had' been at the community college; but the difference was much 
greater for persisters than for dropouts. In other words, the dropouts 
had increased. their quality of effort somewhat, but not nearly enough 
and not nearly as high as the persisters. For example, on the Course 
Learning scale, the percent scoring 26 or higher on the scale was 55% at 
UCLA, for those who had dropped out compared with 37% when they were in 
community college. In contrast, the corresponding percentages for those 
who were persisters at UCLA was 50% when they were ir\ community college, 
increasing the 80% at UCLA. On the library scale the persisters had 65% 
scoring 21 points or above at UCLA, compared with 31% for the dropouts. 
For both groups the percentages represented an increase over what their 
quality of effort in library use had been at the community college— the 
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dropouts having increased from 16% to 31%, and the persisters increasing 
from 28% to 65%. 

These differences between community college and the university are 
also reflected in students ratings of progress toward important objectives. 
In community college, less than a third of the transfer students felt 
they had made very much or quite a bit of progress toward the objective 
of ability to think analytically and logically (34% among those who 
subsequently persisted at UCLA and 27% among those who subsequently 
dropped out). At UCLA, among those who persisted, 85% claimed very much 
or quite a bit of gain, compared with 46% among those who had dropped 
out. 

From these examples, and from many others like them in the complete 
study, two generalizations can be made: first, the quality of academic- 
effort needed for persistence at the university was much higher than the 
quality of effort needed at the community college to become eligible for 
transfer; and second, compared with the students who later dropped out, 
the students who were successful at the university had not only made a 
much larger increase in their prior quality of effort but also had 
reached a much higher absolute level. One of the successful transfer 
students put it this way: "I think it's up to the individual to realize 
that UCLA is not a joke. If he or she bears down and pushes himself or 
herself they will get the most out of what UCLA has to offer. I enjoyed 
it,' and I am not that smart, but I worked hard. And that's what counts." 

This student's comment brings us back to where we started this 
paper: "If students expect to benefit from what this college or university 
has to offer they have to take the initiative." The quality of effort 
scales are, in a sense, measures of initiative. With a few inadvertent 
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exceptions, 'nearly all of the activities in the quality of effort scales 
are essentially voluntary. They are not assigned or required; and it 
way be this very feature that accounts for their significant, relation to 
high quality achievement, and the*ir significant value 1n higher education. 
College is basically a voluntary activity. You don't have to go to - 
college. It may be necessary to go -if you want to "be a, dentist, but no 
one says that you have to be a dentist. After you get to college, you 
don't have to browse in the library, you don't have to make appointments 
to talk with faculty members; you don't have to go to -class, you don't 
have to make outlines from your class notes and readings, you don't have 
to go to concerts, you don't have to work on a committee, you don't have 
to ask other people to read something you have written to see whether it 
was clear to them, you don't have to have serious discussions with 
students whose personal values are very different from yours. By the 
words "don't havfe to" I mean that no one is requiring you to do so -or 
checking up on you to make sure you have. This is why the quality of 
effort, which one might also think of as the quality of initiative, is 
so important at the college level— so highly predictive of achievement, 
so diagnostic for understanding various relationships, and so pervasive 
in the college experience.^ 

How useful it may be to measure "quality of effort" in high school, 

V 

\ 

and in elementary school, may depend on how much opportunity there is 
for "pupil initiative" in those settings. Pupil behavior in the elementary 
school classroom, and the elementary school setting in general, is pri- 
marily planned and controlled by the teacher. It is the teacher who 
decides what activities will be done, and when, and who helps them along 
the way. But, there are surely also opportunities for pupil intitiative, 
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for independence, and for accepting responsibility* Observers and 
researchers who are familiar with elementary schools could devise ways 
to record pupil activities that reflect quality of effort in. using 
elementary school facilities. The evidence revealed by such measurement 
may well be predictive of achievement at the next level of education 
where the opportunities for initiative are greater, and where the connec- 
tion between effort and attainment is probably stronger. In high schools 
more independence and adaptation are required for success. One has 
different teachers for different subjects; one moves from one location 
to another; surveilance of whether one is doing one's work is fiot as 
close as it was in elementary school where one was in the same classroom 
with the same teacher day in and day out. 

It may well be true that whenever education is compulsory, teachers 
feel compelled to plan and manage and monitor the activities of pupils 
in considerable detail. The goal is often mastery, by as many as possibl 
of explicitly described tasks. 

In Jj^cjher education such detailed determination of how students 
spend their time is rarely found. The developmental and educative - 
process from childhood to adolescence to adulthood— which is to say from 
elementary school to secondary school to college--is not only one of 
acquiring more and more knawledge but also one requiring more and more 
initiative. It may be useful to think of the results (success) at each 
level as "readiness to take the next step. 11 

From evidence at the college level that quality of student effort 

predicts progress toward the attainment of important objectives better 

/ 

than any other activity or characteristic, and evidence that the quality 
of effort everted in community college was not enough 1 for persistence 
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and success at the university, readiness to take next step from one edu- 
cational level to the next may be viewed as the individual's readiness 
to invest time and effort in the events and experiences that- are intended 
to facilitate learning and development. ? 

Whatever applicability the concept of quality of effort may have in 
other educational settings its value for higher education is buttressed 
by the eyidence given in this paper. The gist of that evidence is 
really quite simple: college can't give you an education; but if you go 
to college, and fully use the facilities and opportunities it provides, 
you can get an education— indeed, a very good one. 
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